Incremental SVM

1) Approximate incremental SVM: pass through the dataset many times

• Pegasos: select a training batch in each iteration

  • python: https://github.com/ejlb/pegasos

       https://github.com/avaitla/Pegasos
    

  • C: https://www.cs.huji.ac.il/~shais/code/index.html
  • matlab: https://www.mathworks.com/matlabcentral/fileexchange/31401-pegasos-primal-estimated-sub-gradient-solver-for-svm?focused=5188208&tab=function

• sklearn.linear_model: SGD

  clf= sklearn.linear_model.SGDClassifier(learning_rate = 'constant', eta0 = 0.1, shuffle = False, n_iter = 1)
  # get x1, y1 as a new instance
  clf.partial_fit(x1, y1)
  # get x2, y2
  # update accuracy if needed
  clf.partial_fit(x2, y2)

2) Exact incremental or decremental SVM: only pass through the dataset once

• Incremental and Decremental Support Vector Machine Learning
  http://www.isn.ucsd.edu/svm/incremental

• SVM Incremental Learning, Adaptation and Optimization: extend the work above
  matlab: https://github.com/diehl/Incremental-SVM-Learning-in-MATLAB

• Incremental and decremental training for linear classification: extension of liblinear focusing on linear problem
  http://www.csie.ntu.edu.tw/~cjlin/papers/ws/index.html